ohi logo
OHI Science | Citation policy

1 Summary

Spatial data from IUCN and Aquamaps is combined with extinction risk information from IUCN to generate regional scores for the Species subgoal. A region’s status is based upon an area-weighted average of species health across each global reporting region.

From Halpern et al (2012):

The target for the Species sub-goal is to have all species at a risk status of Least Concern. We scaled the lower end of the biodiversity goal to be 0 when 75% species are extinct, a level comparable to the five documented mass extinctions and would constitute a catastrophic loss of biodiversity. The Status of assessed species was calculated as the area- and threat status-weighted average of the number of threatened species within each 0.5 degree grid cell.

Mean risk status per cell:

\[\bar{R}_{cell} = \frac{\displaystyle\sum_{species}(Risk)}{n_{spp}}\]

Mean risk status per region:

\[\bar{R}_{SPP} = \frac{\displaystyle\sum_{cells}(\bar{R}_{cell} * A_{cell} * pA_{cell-rgn})}{A_{rgn}}\]

Species goal model

\[X_{SPP} = \frac{((1 - \bar{R}_{SPP}) - 0.25)}{(1 - 0.25)} * 100%\]

where:

2 Updates from previous assessment

Changes since 2015 SPP subgoal for global OHI:


3 Data Sources

IUCN:

AquaMaps:


4 Methods

4.1 Extract AquaMaps data from .sql files

AquaMaps data for the 2016 assessment was provided as .sql files, as in previous years, that can be used to generate an SQL database. Each line in the .sql is a command to populate the SQL database.

  • Original files extracted from aquamaps_2015_full_dataset_ohi.zip:
    • hcaf_ohi.sql
    • speciesoccursum_ohi.sql
    • hcaf_species_native_ohi.sql
  • Processed files saved to git-annex:
    • hcaf_truncated.csv
    • speciesoccursum.csv
    • hcaf_sp_native_trunc.csv

To extract data, we instead scan each line for CREATE TABLE and INSERT INTO commands to create and save dataframes. Note that the am_extract_2015.R script discards much of the data from these .sqls that is not used within the OHI Species Goal processing (thus “truncated”). This speeds up read time and processing time and avoids parsing issues with some of the rows/columns. Note also that the hcaf_species_native_ohi.sql file does not originally contain LOICZID information; this is added in the extract script, since LOICZID as a cell identifier is faster and less memory intensive than CsquareCode (integer vs character string).

Half-degree cell info from hcaf_truncated.csv (first few rows)
csquarecode loiczid nlimit slimit wlimit elimit centerlat centerlong cellarea oceanarea
5207:363:1 167254 -26.0 -26.5 -73.5 -73.0 -26.25 -73.25 2772.29 2772.29
5207:363:2 167253 -26.0 -26.5 -74.0 -73.5 -26.25 -73.75 2772.29 2772.29
5207:363:3 167974 -26.5 -27.0 -73.5 -73.0 -26.75 -73.25 2760.25 2760.25
5207:363:4 167973 -26.5 -27.0 -74.0 -73.5 -26.75 -73.75 2760.25 2760.25
5207:360:1 167260 -26.0 -26.5 -70.5 -70.0 -26.25 -70.25 2772.29 0.00
Species info from speciesoccursum.csv (first few rows)
speciesid reviewed speccode genus species fbname occurcells kingdom phylum class order family iucn_id iucn_code iucn_version
Fis-156671 null 62612 Abalistes filamentosus null 12 Animalia Chordata Actinopterygii Tetraodontiformes Balistidae null N.E. 2015-2
Fis-53544 1 9 Abalistes stellaris Starry triggerfish 198 Animalia Chordata Actinopterygii Tetraodontiformes Balistidae null N.E. 2015-2
Fis-142700 1 58334 Abalistes stellatus null 235 Animalia Chordata Actinopterygii Tetraodontiformes Balistidae null N.E. 2015-2
Fis-27725 1 10232 Ablabys taenianotus Cockatoo waspfish 59 Animalia Chordata Actinopterygii Scorpaeniformes Tetrarogidae null N.E. 2015-2
Fis-22975 1 972 Ablennes hians Flat needlefish 397 Animalia Chordata Actinopterygii Beloniformes Belonidae null N.E. 2015-2
Species-to-cell lookup from hcaf_sp_native_trunc.csv (first few rows)
speciesid probability loiczid
Fis-29358 1.00 129241
Fis-139729 0.97 129241
Fis-23185 0.81 129241
Fis-29263 1.00 129241
Fis-29290 1.00 129241

4.2 Ingest IUCN species list

To identify appropriate IUCN species for the analysis, we identified all IUCN Red List species whose habitat included “marine” designation. The ingest_iucn.R script scrapes this data directly from the IUCN Red List website.

  • Starting point: Access Red List API at http://api.iucnredlist.org/index/all.csv. This provides a list of all Red List species, including unique IUCN code, taxonomic nomenclature, and Red List extinction risk.
  • For all identified species, we access species-specific info at http://api.iucnredlist.org/details/X/0 where X is the unique IUCN species ID; this page is saved to a cache directory on git-annex as a .htm file. At the same time, we scrape the “habitats” field from the saved page using XML tags.
  • All species with “marine” habitat are then scraped to find details on population trend and subpopulation ID numbers.
  • After some cleaning of scientific names (accents, spaces, html tags, etc) the file is saved to git-annex for later use.

Processed files, saved to git-annex/globalprep/spp_ico/v201x/int: * spp_iucn_all.csv - full list of IUCN species pulled from web, some cleaning. * spp_iucn_habitats.csv - list of IUCN species (by iucn_sid) and corresponding habitat. * spp_iucn_marine.csv - prepped list: cleaned marine list with subpops and trends.

IUCN marine species list and info from spp_iucn_marine.csv (first few rows)
sciname class order family genus species authority iucn_sid modified_year category criteria habitat popn_trend subpop_sid parent_sid
Abantennarius analis ACTINOPTERYGII LOPHIIFORMES ANTENNARIIDAE Abantennarius analis (Schultz, 1957) 155277 2010 LC NA Marine Unknown NA NA
Ablennes hians ACTINOPTERYGII BELONIFORMES BELONIDAE Ablennes hians (Valenciennes, 1846) 13486514 2015 LC NA Marine Unknown NA NA
Ablennes pacificus ACTINOPTERYGII BELONIFORMES BELONIDAE Ablennes pacificus (Valenciennes, 1846) 13486514 2015 LC NA Marine Unknown NA NA
Aboma etheostoma ACTINOPTERYGII PERCIFORMES GOBIIDAE Aboma etheostoma Jordan & Starks, 1895 183435 2010 DD NA Marine Unknown NA NA
Aboma snyderi ACTINOPTERYGII PERCIFORMES GOBIIDAE Aboma snyderi (Temminck & Schlegel, 1845) 181137 2012 LC NA Marine Unknown NA NA

4.3 Generate full species lookup table

Having processed AquaMaps and IUCN species raw data, we can now prepare a full combined list of all species to be included in the OHI SPP goal. The function create_spp_master_lookup() creates the full lookup table:

  • Load AquaMaps species info from speciesoccursum.csv
    • Verify sciname field; the verify_scinames() function uses taxize::gnr_resolve() to compare AM scinames to accepted names from Encyclopedia of Life and NCBI databases - this helps resolve differences in naming conventions and species aliases.
  • Load IUCN species info from spp_iucn_marine.csv
    • Verify sciname field as above.
  • Join the two species lists by verified scientific names as the best common identifier.
  • Create an extinction risk category field for each row, using IUCN category, or failing that the category listed in AquaMaps; failing that, assign NA.

At this point, the list is all AquaMaps species and all marine-identified IUCN species. Next, identify the source of spatial distribution data for each species, if available.

  • From the downloaded IUCN shapefiles, determine which species have spatial data.
  • Depending on preference and availability of spatial source (AquaMaps or IUCN), tag each row with a specific spatial_source (“am” or “iucn”).
  • Follow a similar process for IUCN species included in the BirdLife International geodatabase (tag as “iucn-bli”).

Now having identified spatial source availability and preference for all species on the list:

  • Clean out duplicated species
  • Assign numeric values to population Red List category and population trend.
  • Save result to git-annex/globalprep/spp_ico/v2016/int/spp_all_raw.csv
  • Filter out subpop species rows (species with a non-NA parent_sid) that do not have a named subpopulation location (i.e. iucn_subpop field is NA, instead of the name of a subpopulation)
  • Save final list to git-annex/globalprep/spp_ico/v2016/int/spp_all_cleaned.csv
Combined species list from spp_all_cleaned.csv (first few rows)
am_sid am_cat sciname iucn_sid pop_trend pop_cat spp_group id_no iucn_subpop spatial_source cat_score trend_score
Fis-26169 LC Apolemichthys trimaculatus 165835 Stable LC ANGELFISH 165835 NA iucn 0 0
Fis-28014 LC Apolemichthys xanthotis 165853 Stable LC ANGELFISH 165853 NA iucn 0 0
Fis-28015 LC Apolemichthys xanthurus 165844 Unknown LC ANGELFISH 165844 NA iucn 0 NA
Fis-27342 LC Centropyge acanthops 155083 Stable LC ANGELFISH 155083 NA iucn 0 0
Fis-22168 LC Centropyge argi 165837 Unknown LC ANGELFISH 165837 NA iucn 0 NA

4.4 Spatialize species information using AquaMaps and IUCN spatial data

4.4.1 Extract IUCN polygons to half-degree cells

We extract IUCN polygon presence to the same half-degree cells as AquaMaps to simplify the analysis. * The spp_all species list includes a field spp_group that identifies which shapefile contains the spatial information for a given species. * for each species group, specific species are identified by comparing iucn_sid from dataframe to id_no within the shapefile. * Extract loiczid cell IDs for each species within each species group. Save a .csv file for that group, with fields: * sciname | iucn_sid | presence | subpop | LOICZID | prop_area * presence codes: 1 extant; 2 prob extant (discontinued); 3 Possibly Extant; 4 Possibly Extinct; 5 Extinct (post 1500); 6 Presence Uncertain * NOTE: this takes a long time - multiple hours for some of the shape files.
* by passing a filtered data frame to the function, you can focus the process only on new or updated shapefiles * reload = FALSE allows the function to skip extraction on groups with files already present. Set to TRUE if you need to extract an updated shapefile (or change the shapefile name, or delete the previous extraction…)

4.4.2 Generate cell-by-cell summary of species

For each half-degree cell, tally up the number of species present and determine a mean species risk value and population trend value for the cell.

  • At this point, the species-cell lists are filtered to species with valid extinction risk categories - i.e. not DD and not NA.
  • Since this averaging is done for each data set separately, we also track the species count per cell used to determine both the risk and the trend (separately, since many species with a risk value have no trend information, i.e. NA). These counts are used to weight the values when the two are combined.
  • Data-set specific idiosyncracies:
    • For AquaMaps, we apply a threshold to set the minimum probability of occurrence that determines species “presence.”
    • For IUCN, no threshold is needed; but the shapefiles include a “presence” attribute in which a value of 5 indicates a region in which a subpopulation has become extinct. We use this to manually reset local extinction risk and trend to EX and NA respectively.
    • Note that for IUCN, we determine the proportional area when extracting polygons; currently we just consider any presence to fill the cell (similar to assuming even a low AquaMaps probability to indicate presence within the entire cell).
AquaMaps cell summary (first few rows)
loiczid mean_cat_score mean_pop_trend_score n_cat_species n_trend_species source
8205 0 NaN 1 0 aquamaps
8206 0 NaN 1 0 aquamaps
8207 0 NaN 1 0 aquamaps
8209 0 NaN 1 0 aquamaps
8210 0 NaN 1 0 aquamaps
8211 0 NaN 1 0 aquamaps
IUCN cell summary (first few rows)
loiczid mean_cat_score mean_pop_trend_score n_cat_species n_trend_species source
1 0.4 -0.5 1 1 iucn
2 0.4 -0.5 1 1 iucn
3 0.4 -0.5 1 1 iucn
4 0.4 -0.5 1 1 iucn
5 0.4 -0.5 1 1 iucn
6 0.4 -0.5 1 1 iucn

4.5 Summarize status and trend by region

Finally we take the two cell-by-cell summaries and combine, using a species-count weighting to determine the mean category and trend per cell. Cells are aggregated to regions, to calculate an area-weighted regional mean category, trend, and status.

These are then saved to status and trend layer outputs for global (shown in table) as well as 3 nautical mile, Antarctic, and High Seas regions.

4.6 Determine species by region

The calc_rgn_spp() function takes in lookup tables of species by cell (for both IUCN and AM), a cell-to-region lookup, and a species info lookup. From this it generates a list of which species occur in which regions, including species information.

Species by region - first few rows
iucn_sid am_sid sciname pop_cat pop_trend spatial_source rgn_id rgn_name n_cells presence n_spp_rgn
NA Fis-156671 Abalistes filamentosus NA NA am 210 Japan 194 NA 11503
NA Fis-156671 Abalistes filamentosus NA NA am 20 South Korea 35 NA 5861
NA Fis-156671 Abalistes filamentosus NA NA am 255 DISPUTED 145 NA 18267
NA Fis-156671 Abalistes filamentosus NA NA am 209 China 151 NA 10906
NA Fis-156671 Abalistes filamentosus NA NA am 14 Taiwan 67 NA 10938
NA Fis-156671 Abalistes filamentosus NA NA am 207 Vietnam 113 NA 10348

4.7 Comparing scenarios and data

The following plots compare the status scores generated for the 2015 assessment to those generated for 2016.

  • The first compares the released scores for 2015 to the proposed scores for 2016. This changes both the datasets and the methodology between the two axes.
    • 2015 scores used data from 2014 (d2014) for both AquaMaps and IUCN, and scripts from the v2015 scenario.
    • 2016 scores use data from 2015 (d2015) for both AquaMaps and IUCN, and scripts from the v2016 scenario.
  • The first compares regenerated scores for 2015 to the proposed scores for 2016. This keeps the methodology largely the same between the two axes (except as noted below), and compares the changes in data.
    • 2015 scores used data from 2014 (d2014) for both AquaMaps and IUCN.
    • 2016 scores use data from 2015 (d2015) for both AquaMaps and IUCN.
    • Both scores use scripts from the v2016 scenario. Methodology exceptions:
      • AquaMaps for d2014 is processed with a 40% threshold (as in v2015) while d2015 is processed with a 0% threshold.
      • IUCN species-cell lookup for d2014 was generated using v2015 scripts, which relied on the sciname field for joining. I’ve added a step to add iucn_sid values according to sciname, to allow the species-cell lookup to work with the v2016 scripts.

5 Provenance

## pre-main prep time: 3 ms

Provenance summary for run 6 (99eedd8): standard run (Fri May 6 13:50:44 2016)
file_name file_dir filetype uncomm_chgs commit_hash
am_cells_spp_prob0.csv Mazu:/git-annex/globalprep/spp_ico/v2016/int input TRUE NA
cellID_regions_gcs_aq.csv Mazu:/git-annex/globalprep/spp_ico/rgns input TRUE NA
cellID_regions_gcs_global.csv Mazu:/git-annex/globalprep/spp_ico/rgns input TRUE NA
cellID_regions_gcs_global.csv Mazu:/git-annex/globalprep/spp_ico/rgns input TRUE NA
cellID_regions_gcs_hs.csv Mazu:/git-annex/globalprep/spp_ico/rgns input TRUE NA
cellID_rgn_offshore3nm_gcs_global.csv Mazu:/git-annex/globalprep/spp_ico/rgns input TRUE NA
cell_spp_summary_by_loiczid.csv ~/github/ohiprep/globalprep/spp_ico/v2016/summary input FALSE 5f8aba9
cell_spp_summary_by_loiczid.csv ~/github/ohiprep/globalprep/spp_ico/v2016/summary input FALSE 5f8aba9
cell_spp_summary_by_loiczid.csv ~/github/ohiprep/globalprep/spp_ico/v2016/summary input FALSE 5f8aba9
cell_spp_summary_by_loiczid_d14v16.csv ~/github/ohiprep/globalprep/spp_ico/v2016/summary input FALSE 5f8aba9
hcaf.csv Mazu:/git-annex/globalprep/_raw_data/aquamaps/d2015/../d2014/tables input TRUE NA
hcaf_sp_native_trunc.csv Mazu:/git-annex/globalprep/_raw_data/aquamaps/d2015/csv input TRUE NA
hcaf_truncated.csv Mazu:/git-annex/globalprep/_raw_data/aquamaps/d2015/csv input TRUE NA
iucn_cells_spp.csv Mazu:/git-annex/globalprep/spp_ico/v2016/int input TRUE NA
iucn_cells_spp.csv Mazu:/git-annex/globalprep/spp_ico/v2015/int input TRUE NA
ohi_hcaf_species_native.csv Mazu:/git-annex/globalprep/_raw_data/aquamaps/d2015/../d2014/tables input TRUE NA
rgn_spp_gl.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output input FALSE b21fe29
rgn_summary_3nm.csv ~/github/ohiprep/globalprep/spp_ico/v2016/summary input FALSE 07ea4c3
rgn_summary_aq.csv ~/github/ohiprep/globalprep/spp_ico/v2016/summary input FALSE 5f8aba9
rgn_summary_gl.csv ~/github/ohiprep/globalprep/spp_ico/v2016/summary input FALSE 07ea4c3
rgn_summary_gl_d14v16.csv ~/github/ohiprep/globalprep/spp_ico/v2016/summary input FALSE 07ea4c3
rgn_summary_hs.csv ~/github/ohiprep/globalprep/spp_ico/v2016/summary input FALSE 07ea4c3
speciesoccursum.csv Mazu:/git-annex/globalprep/_raw_data/aquamaps/d2015/csv input TRUE NA
spp_all_cleaned.csv Mazu:/git-annex/globalprep/spp_ico/v2016/int input TRUE NA
spp_all_cleaned.csv Mazu:/git-annex/globalprep/spp_ico/v2016/int input TRUE NA
spp_all.csv Mazu:/git-annex/globalprep/spp_ico/v2015/int input TRUE NA
spp_iucn_all.csv Mazu:/git-annex/globalprep/spp_ico/v2016/int input TRUE NA
spp_iucn_habitats.csv Mazu:/git-annex/globalprep/spp_ico/v2016/int input TRUE NA
spp_iucn_marine.csv Mazu:/git-annex/globalprep/spp_ico/v2016/int input TRUE NA
spp_status_global.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output input FALSE 07ea4c3
spp_status_global.csv ~/github/ohiprep/globalprep/spp_ico/v2015/data input FALSE aaa53b5
spp_status_global_d14v16.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output input FALSE 07ea4c3
spp_sum_am_cells.csv ~/github/ohiprep/globalprep/spp_ico/v2016/summary input FALSE 07ea4c3
spp_sum_am_cells_d14v16.csv ~/github/ohiprep/globalprep/spp_ico/v2016/summary input FALSE 5f8aba9
spp_sum_iucn_cells.csv ~/github/ohiprep/globalprep/spp_ico/v2016/summary input FALSE b21fe29
spp_sum_iucn_cells_d14v16.csv ~/github/ohiprep/globalprep/spp_ico/v2016/summary input FALSE 5f8aba9
spp_trend_global.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output input FALSE 07ea4c3
spp_trend_global.csv ~/github/ohiprep/globalprep/spp_ico/v2015/data input FALSE aaa53b5
spp_trend_global_d14v16.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output input FALSE 07ea4c3
trend_and_subpops.csv Mazu:/git-annex/globalprep/spp_ico/v2016/int input TRUE NA
rgn_spp_3nm.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output output FALSE 5f8aba9
rgn_spp_aq.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output output FALSE 5f8aba9
rgn_spp_gl.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output output FALSE b21fe29
rgn_spp_hs.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output output FALSE 5f8aba9
spp_iucn_marine.csv Mazu:/git-annex/globalprep/spp_ico/v2016/int output TRUE NA
spp_status_3nm.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output output FALSE 07ea4c3
spp_status_aq.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output output FALSE 5f8aba9
spp_status_global.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output output FALSE 07ea4c3
spp_status_global_d14v16.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output output FALSE 07ea4c3
spp_status_hs.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output output FALSE 07ea4c3
spp_sum_am_cells_d14v16.csv ~/github/ohiprep/globalprep/spp_ico/v2016/summary output FALSE 5f8aba9
spp_sum_iucn_cells_d14v16.csv ~/github/ohiprep/globalprep/spp_ico/v2016/summary output FALSE 5f8aba9
spp_trend_3nm.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output output FALSE 07ea4c3
spp_trend_aq.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output output FALSE 5f8aba9
spp_trend_global.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output output FALSE 07ea4c3
spp_trend_global_d14v16.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output output FALSE 07ea4c3
spp_trend_hs.csv ~/github/ohiprep/globalprep/spp_ico/v2016/output output FALSE 07ea4c3
data_prep_SPP.Rmd ~/github/ohiprep/globalprep/spp_ico/v2016 parent_script NA NA
ingest_iucn.R ~/github/ohiprep/globalprep/spp_ico/R sourced_script FALSE 76b0910
layer_prep_spp_3nm.R ~/github/ohiprep/globalprep/spp_ico/v2016 sourced_script FALSE 07ea4c3
layer_prep_spp_global.R ~/github/ohiprep/globalprep/spp_ico/v2016 sourced_script TRUE 07ea4c3
layer_prep_spp_hs_aq.R ~/github/ohiprep/globalprep/spp_ico/v2016 sourced_script FALSE 07ea4c3
prep_spp_list.R ~/github/ohiprep/globalprep/spp_ico/v2016 sourced_script FALSE 76b0910
score_plot_spp.R ~/github/ohiprep/globalprep/spp_ico/v2016 sourced_script FALSE 07ea4c3
spp_fxn.R ~/github/ohiprep/globalprep/spp_ico/v2016 sourced_script FALSE 07ea4c3